<p>In this article, we seek to investigate bias and quality of so called “state-of-the art” Artificial Intelligence (AI) driven language models under the lens of geographical regions. We assess OpenAI’s ChatGPT model and its responses to a diverse group of testers spanning North America, Europe, and Africa. In an era where AI-driven language models are increasingly influencing decision-making and shaping opinions, it is crucial to assess their advice and guidance, particularly on issues of public interest. This study employs a non-representative large sample of respondents, drawn from various walks of life across the three continents. While no representative sampling techniques were employed, the diverse geographical and cultural backgrounds of the participants provide a rich and varied dataset that is leveraged in a rigorously comparative analysis. Our research focuses on ChatGPT’s responses to critical topics in public administration including paying taxes, Indigenous Peoples’ rights, Indigenous Peoples’ self-determination, and whistleblowing, among others. Using advanced machine-learning techniques, we assess the quality and topical orientation of ChatGPT’s advice on these selected topics and conduct regional comparisons. This comparative approach helps flag regional variations in the model’s responses and provides evidence ChatGPT’s answers vary with geographical locations of users. Furthermore, we integrate conceptual insights from the field of public administration to situate our study in a discipline that has seen increased interest in AI adoption. With the diverse, thoroughly selected group of multiple respondents, this study contributes to understanding the implications of AI-driven language models and their limitations when looked at under regional perspectives. We believe our findings will contribute towards the development of region-fair AI-driven conversational agents and offer valuable insights for policymakers, educators, and users.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

The devil is in the details: using machine-learning to scrutinize “state-of-the-art” language models’ responses to public inquiries across 3 continents

  • Stany Nzobonimpa,
  • Jean-François Savard

摘要

In this article, we seek to investigate bias and quality of so called “state-of-the art” Artificial Intelligence (AI) driven language models under the lens of geographical regions. We assess OpenAI’s ChatGPT model and its responses to a diverse group of testers spanning North America, Europe, and Africa. In an era where AI-driven language models are increasingly influencing decision-making and shaping opinions, it is crucial to assess their advice and guidance, particularly on issues of public interest. This study employs a non-representative large sample of respondents, drawn from various walks of life across the three continents. While no representative sampling techniques were employed, the diverse geographical and cultural backgrounds of the participants provide a rich and varied dataset that is leveraged in a rigorously comparative analysis. Our research focuses on ChatGPT’s responses to critical topics in public administration including paying taxes, Indigenous Peoples’ rights, Indigenous Peoples’ self-determination, and whistleblowing, among others. Using advanced machine-learning techniques, we assess the quality and topical orientation of ChatGPT’s advice on these selected topics and conduct regional comparisons. This comparative approach helps flag regional variations in the model’s responses and provides evidence ChatGPT’s answers vary with geographical locations of users. Furthermore, we integrate conceptual insights from the field of public administration to situate our study in a discipline that has seen increased interest in AI adoption. With the diverse, thoroughly selected group of multiple respondents, this study contributes to understanding the implications of AI-driven language models and their limitations when looked at under regional perspectives. We believe our findings will contribute towards the development of region-fair AI-driven conversational agents and offer valuable insights for policymakers, educators, and users.