Multitask generalization Grounded on minimal supervision of knowledge-generalizable capability has emerged as part of the pursuit of an Artificial General Intelligence (AGI). Few-shot learning: a regime where multi-task models solve tasks with as few as tens of examples has been identified as one of the key signs of AGI potential. In this paper, the few shot and zero shot capabilities of transformer based models will be explored through real world data in AG News dataset. It used the facebook/bart-base model to assess the level of confidence and success with which the model identifies unseen news articles in pre-defined classes; it has a zero-shot classification pipeline. The visualization of results in the form of bar plots indicates a moderate discriminatory power of the model, yet, confidence levels may be quite blankly distributed within several classes indicating less robustness of the models in nature. The results also depict the current capabilities and limitations of LLMs regarding the approximate AGI and suggest that a more sophisticated prompt engineering strategy and model alignment hold the key to making more general reasoning possible.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Evaluating Few-Shot Learning Capabilities of Large Language Models as a Step Toward Artificial General Intelligence

  • Koppula ChinaBusi,
  • Kondragunta Rama Krishnaiah,
  • Sreepathi Ramesh Babu,
  • A. Silpa,
  • Bhanu Prakash Battula,
  • Kuppani Sathish

摘要

Multitask generalization Grounded on minimal supervision of knowledge-generalizable capability has emerged as part of the pursuit of an Artificial General Intelligence (AGI). Few-shot learning: a regime where multi-task models solve tasks with as few as tens of examples has been identified as one of the key signs of AGI potential. In this paper, the few shot and zero shot capabilities of transformer based models will be explored through real world data in AG News dataset. It used the facebook/bart-base model to assess the level of confidence and success with which the model identifies unseen news articles in pre-defined classes; it has a zero-shot classification pipeline. The visualization of results in the form of bar plots indicates a moderate discriminatory power of the model, yet, confidence levels may be quite blankly distributed within several classes indicating less robustness of the models in nature. The results also depict the current capabilities and limitations of LLMs regarding the approximate AGI and suggest that a more sophisticated prompt engineering strategy and model alignment hold the key to making more general reasoning possible.