Evaluating Few-Shot Learning Capabilities of Large Language Models as a Step Toward Artificial General Intelligence
摘要
Multitask generalization Grounded on minimal supervision of knowledge-generalizable capability has emerged as part of the pursuit of an Artificial General Intelligence (AGI). Few-shot learning: a regime where multi-task models solve tasks with as few as tens of examples has been identified as one of the key signs of AGI potential. In this paper, the few shot and zero shot capabilities of transformer based models will be explored through real world data in AG News dataset. It used the facebook/bart-base model to assess the level of confidence and success with which the model identifies unseen news articles in pre-defined classes; it has a zero-shot classification pipeline. The visualization of results in the form of bar plots indicates a moderate discriminatory power of the model, yet, confidence levels may be quite blankly distributed within several classes indicating less robustness of the models in nature. The results also depict the current capabilities and limitations of LLMs regarding the approximate AGI and suggest that a more sophisticated prompt engineering strategy and model alignment hold the key to making more general reasoning possible.