Scholarlink

<p>We investigate the theoretical and empirical foundations of applying Large Language Models (LLMs) to structured tabular data for safety-critical classification, specifically road fatality prediction. While transformer architectures present a paradigm shift in predictive modeling, their application to tabular data suffers from a fundamental impedance mismatch. We formalize this mismatch through the lens of information theory, modeling the performance degradation of LLMs on symbolic formats (e.g., JSON, XML) as a function of <i>Attention Dilution</i> and high Kullback–Leibler (KL) divergence from pre-training distributions. Through a rigorous 10-format ablation study, we prove that Natural Language Serialization (NLS) mathematically minimizes this information bottleneck, aligning structured tabular data with the syntactic inductive biases of the attention mechanism. Leveraging this optimal representation, we systematically evaluate a suite of transformer architectures against state-of-the-art tabular deep learning (TabTransformer) and gradient boosting frameworks. ALBERT emerges as the definitive empirical champion, achieving a new state-of-the-art (SOTA) test accuracy of 87.19% and an AUROC of 0.975 on the UK Fatal Accident Dataset. The statistical significance of ALBERT’s superiority over all tabular baselines is confirmed via McNemar’s Tests (<InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(p &lt; 0.001\)</EquationSource> </InlineEquation>). Furthermore, through a 21-strategy complexity ablation study involving four leading LLMs, we mathematically demonstrate that complex ensemble mechanisms fail to yield super-additive performance gains over the standalone ALBERT model, establishing a “Simplicity as a Feature” paradigm. We validate the cross-domain generalizability of this framework on the US Accidents dataset and ensure regulatory transparency through a deep explainability suite comprising SHAP, LIME, and Architectural Attention Heatmaps. This study provides a mathematically mature blueprint for maximizing LLM performance in specialized tabular domains.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

On the Representation of Tabular Data for Explainable LLM-Based Road Fatality Prediction

  • 期刊论文

  • 发表日期：

    2026年5月20日

查看全文
  • Umar Hasan,
  • Riasat Khan

摘要

We investigate the theoretical and empirical foundations of applying Large Language Models (LLMs) to structured tabular data for safety-critical classification, specifically road fatality prediction. While transformer architectures present a paradigm shift in predictive modeling, their application to tabular data suffers from a fundamental impedance mismatch. We formalize this mismatch through the lens of information theory, modeling the performance degradation of LLMs on symbolic formats (e.g., JSON, XML) as a function of Attention Dilution and high Kullback–Leibler (KL) divergence from pre-training distributions. Through a rigorous 10-format ablation study, we prove that Natural Language Serialization (NLS) mathematically minimizes this information bottleneck, aligning structured tabular data with the syntactic inductive biases of the attention mechanism. Leveraging this optimal representation, we systematically evaluate a suite of transformer architectures against state-of-the-art tabular deep learning (TabTransformer) and gradient boosting frameworks. ALBERT emerges as the definitive empirical champion, achieving a new state-of-the-art (SOTA) test accuracy of 87.19% and an AUROC of 0.975 on the UK Fatal Accident Dataset. The statistical significance of ALBERT’s superiority over all tabular baselines is confirmed via McNemar’s Tests ( \(p < 0.001\) ). Furthermore, through a 21-strategy complexity ablation study involving four leading LLMs, we mathematically demonstrate that complex ensemble mechanisms fail to yield super-additive performance gains over the standalone ALBERT model, establishing a “Simplicity as a Feature” paradigm. We validate the cross-domain generalizability of this framework on the US Accidents dataset and ensure regulatory transparency through a deep explainability suite comprising SHAP, LIME, and Architectural Attention Heatmaps. This study provides a mathematically mature blueprint for maximizing LLM performance in specialized tabular domains.