Scholarlink

<p>Public complaint classification is essential for effective municipal governance, yet remains challenging due to heterogeneous structured data, severe class imbalance, and limited ability of structured features to capture underlying representation. Unlike the existing studies that relied solely on the original features for end-to-end prediction, the contribution of this study lied in transforming large language models (LLMs) generated reasoning under few-shot learning into an explicit feature engineering step, where reasoning outputs are integrated with structured data and embedded to enrich representations. Extensive experiments were conducted on a real-world municipal complaint dataset containing over 48103 instances across 180 complaint subcategories. The best-performing configuration which combined Qwen3-14B, Paraphrase-Multilingual-Mpnet-Base-V2 embeddings, and Logistic Regression, achieved an average score of 44.8%, with an accuracy of 54%, an F1-score of 23.9%, and an Area Under the Curve of 92%. Moreover, error analysis revealed low class-wise error rates alongside reduced false positive and false negative rates, indicating balanced control over both false alarms and missed detections. The Shapiro-Wilk test rejected normality (statistic <InlineEquation ID="IEq1"><EquationSource Format="TEX">\(\ge \)</EquationSource></InlineEquation> 0.873, <i>p</i> &lt; 0.05), while the Friedman test indicated significant differences across models (statistic <InlineEquation ID="IEq2"><EquationSource Format="TEX">\(\ge \)</EquationSource></InlineEquation> 182.855, <i>p</i> &lt; 0.05), further supported by non-overlapping confidence intervals with F1-score gains from 7.3–7.8% to 12–12.7%. Ablation studies highlight the critical role of each component of proposed framework, improving accuracy from 34.4 to 37.5% and F1-score from 12.5 to 14.1%. Overall, these findings demonstrated that LLM-generated reasoning effectively enhanced structured representations and predictive performance of existing Machine Learning classifiers in public complaint classification.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Enriching structured dataset via reasoning embeddings with few-shot learning for enhancing complaint classification

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月8日

查看全文
  • Theng-Jia Law,
  • Choo-Yee Ting

摘要

Public complaint classification is essential for effective municipal governance, yet remains challenging due to heterogeneous structured data, severe class imbalance, and limited ability of structured features to capture underlying representation. Unlike the existing studies that relied solely on the original features for end-to-end prediction, the contribution of this study lied in transforming large language models (LLMs) generated reasoning under few-shot learning into an explicit feature engineering step, where reasoning outputs are integrated with structured data and embedded to enrich representations. Extensive experiments were conducted on a real-world municipal complaint dataset containing over 48103 instances across 180 complaint subcategories. The best-performing configuration which combined Qwen3-14B, Paraphrase-Multilingual-Mpnet-Base-V2 embeddings, and Logistic Regression, achieved an average score of 44.8%, with an accuracy of 54%, an F1-score of 23.9%, and an Area Under the Curve of 92%. Moreover, error analysis revealed low class-wise error rates alongside reduced false positive and false negative rates, indicating balanced control over both false alarms and missed detections. The Shapiro-Wilk test rejected normality (statistic \(\ge \) 0.873, p < 0.05), while the Friedman test indicated significant differences across models (statistic \(\ge \) 182.855, p < 0.05), further supported by non-overlapping confidence intervals with F1-score gains from 7.3–7.8% to 12–12.7%. Ablation studies highlight the critical role of each component of proposed framework, improving accuracy from 34.4 to 37.5% and F1-score from 12.5 to 14.1%. Overall, these findings demonstrated that LLM-generated reasoning effectively enhanced structured representations and predictive performance of existing Machine Learning classifiers in public complaint classification.