Foundation models for tabular data offer in-context learning for classification and regression problems without training a specific model for each dataset, as required by the prevailing tree ensemble methods. Recently, a new version of the TabPFN foundation model was introduced, extending support to regression tasks on small datasets. In addition, four more computationally intensive variants were released, aiming to improve accuracy and extend applicability to larger datasets. In this study, we compared TabPFN and its four variants to CARTE, a previously proposed foundation model, which reported outperforming the previous version of TabPFN. To assess real-world accuracy, we used seven proprietary datasets that were not available during the pre-training. Our results showed that TabPFN outperformed CARTE across the datasets. Interestingly, TabPFN also outperformed its more complex variants in most cases, without performing their heavier computations. As tabular datasets are extremely diverse, we believe that evaluating such foundation models on proprietary real-world data may offer complementary insights to studies based solely on public datasets.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Evaluating TabPFN for Real-World Small Dataset Regressions

  • Shir Cohen,
  • Daniel Haim Yehezkel,
  • Amitai Armon

摘要

Foundation models for tabular data offer in-context learning for classification and regression problems without training a specific model for each dataset, as required by the prevailing tree ensemble methods. Recently, a new version of the TabPFN foundation model was introduced, extending support to regression tasks on small datasets. In addition, four more computationally intensive variants were released, aiming to improve accuracy and extend applicability to larger datasets. In this study, we compared TabPFN and its four variants to CARTE, a previously proposed foundation model, which reported outperforming the previous version of TabPFN. To assess real-world accuracy, we used seven proprietary datasets that were not available during the pre-training. Our results showed that TabPFN outperformed CARTE across the datasets. Interestingly, TabPFN also outperformed its more complex variants in most cases, without performing their heavier computations. As tabular datasets are extremely diverse, we believe that evaluating such foundation models on proprietary real-world data may offer complementary insights to studies based solely on public datasets.