The digital interface of global events has become a key space for political mobilization and public engagement. In contexts like the Conferences of the Parties (COPs), official websites are not mere institutional repositories, they are gateways for international visibility, citizen participation, and access to critical climate information. With Belém (Brazilian Amazon) set to host COP30, attention turns to whether its digital infrastructure supports inclusive and effective user experiences. This study asks: To what extent are Generative Artificial Intelligence tools capable of identifying relevant usability issues on these platforms? And how can their analyses support Human-Computer Interaction practices that are more accessible and equitable? To address these questions, we conducted a comparative analysis of the COP30 and COP29 websites using four GAIs (ChatGPT, DeepSeek, Gemini, and Copilot) activated with a standardized usability evaluation prompt. A Directed Categorical Content Analysis, based on ISO 9241-210 criteria, enabled both quantitative and qualitative assessments. Results indicate that GAIs show promise in detecting structural and navigational issues, such as deep menu hierarchies and lack of accessibility features. However, limitations were observed, including inconsistent evaluations of user satisfaction and failure to assess technical accessibility aspects. The findings highlight the potential of GAIs as complementary tools in early-stage usability reviews, while also revealing critical gaps that demand human oversight and contextual sensitivity.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Generative AI as a Usability Evaluator: Hype or Help?

  • Danilo T. Lima,
  • Alana M. Medeiros,
  • Rita de Cássia R. Paulino,
  • Marcos César R. Seruffo

摘要

The digital interface of global events has become a key space for political mobilization and public engagement. In contexts like the Conferences of the Parties (COPs), official websites are not mere institutional repositories, they are gateways for international visibility, citizen participation, and access to critical climate information. With Belém (Brazilian Amazon) set to host COP30, attention turns to whether its digital infrastructure supports inclusive and effective user experiences. This study asks: To what extent are Generative Artificial Intelligence tools capable of identifying relevant usability issues on these platforms? And how can their analyses support Human-Computer Interaction practices that are more accessible and equitable? To address these questions, we conducted a comparative analysis of the COP30 and COP29 websites using four GAIs (ChatGPT, DeepSeek, Gemini, and Copilot) activated with a standardized usability evaluation prompt. A Directed Categorical Content Analysis, based on ISO 9241-210 criteria, enabled both quantitative and qualitative assessments. Results indicate that GAIs show promise in detecting structural and navigational issues, such as deep menu hierarchies and lack of accessibility features. However, limitations were observed, including inconsistent evaluations of user satisfaction and failure to assess technical accessibility aspects. The findings highlight the potential of GAIs as complementary tools in early-stage usability reviews, while also revealing critical gaps that demand human oversight and contextual sensitivity.