Online petition systems serve as essential crowd-sourcing mechanisms that enable governments to collect public suggestions on policy matters. Nonetheless, the dependence on manual petition review processes presents challenges for administrators of such systems, particularly in terms of extended response times and delayed feedback, which may discourage active public participation. This study presents an interpretable automated classification approach designed to assess the admissibility of petitions, thereby assisting administrative reviews by providing prompt and transparent feedback to petitioners. The proposed approach uses a fine-tuned BERT model, augmented by principal component analysis (PCA) for dimensionality reduction. The model is further optimized using an XGBoost classifier, to categorize petitions submitted to Taiwan’s JOIN platform into three administrative categories. To address interpretability concerns commonly associated with machine learning-based systems, two explainable AI methods are incorporated: BERT’s Self-Attention mechanism for global textual analysis and SHAP for localized word-level interpretability. Experimental results demonstrate that the developed system achieves a classification accuracy of 67%, surpassing a standalone fine-tuned BERT classifier by 4%, while also reducing model training time. The combination of classification optimization and interpretability analysis holds significant potential for improving the efficiency and transparency of online petition systems.