A Comparative Study: Reasoning with Small T5-Based Models on Abstract Reasoning Corpus (ARC)
摘要
Small Language Models (SLMs) have drawn significant attention due to their computational efficiency under resource constraints, yet their performance in complex reasoning remains underexplored. This study assesses the reasoning capabilities of four T5-based SLMs T5-tiny, T5-small, T5-medium, and T5-FLAN using the ARC dataset to examine how effectively these models learn and applying patterns in tasks in which requiring reasoning skills. Multiple experiments were conducted employing two input strategies: integrating a helping prompt during training to guide reasoning, and feeding directly pattern-based inputs without additional prompts. Preliminary results show that the trained small language models using ARC dataset can notably enhance the ability of these models to identify and leverage relevant patterns of reasoning performance. However, perceiving help from prompt is less significant in terms of quality and data accuracy. Among the four T5 variants, larger models exhibited more consistent gains, underscoring the importance of model complexity and capacity in leveraging prompts for reasoning tasks. These findings highlight both the potential and limitations of SLMs in addressing nuanced reasoning challenges and indicating that further research is required to refine more approaches for scaling and generalizing additional methods to more demanding datasets and a wider range of tasks.