Comparative Analysis of Different Generative AI Models For Scholarly Articles
摘要
In the world of academic research support and information retrieval, the advancement of Generative AI models has opened up novel ways for efficient literature exploration. This research conducts a comparative analysis of three of the best and most prominent AI models—ChatGPT(GPT-3.5), Google Gemini (formerly known as Google Bard), Microsoft Copilot, and ChatGPT(GPT-4) in suggesting academic research papers on blockchain technology. Additionally, Meta AI, a chatbot embedded inside of WhatsApp and accessible to select users (as of April 2024) and Claude 3.5 Sonnet, a capable LLM out of the Claude family of LLMs by Anthropic was included in our analysis as well. Each Large Language Model (LLM) was tasked with the generation of 100 research papers related to blockchain technology. Subsequently, the suggested papers were rigorously cross-checked with the results found in Google Scholar(results sorted by relevance) to validate their existence in the academic literature. This methodology is aimed at assessing the precision and accuracy of AI-driven research assistance-based tools. Our findings shed light on the distinct performance variations among the various Large Language Models (LLMs), underscoring their potential for aiding researchers in literature review tasks. Notably, our analysis contributes valuable insights into the evolving landscape of AI-powered research tools and their implementations in the academic workflow.