Scholarlink

Contraception counselling is often limited by time and lack of personalisation, prompting women to seek advice from AI models. This study evaluated the accuracy, completeness, and relevance of responses generated by Google Gemini, DeepSeek, and ChatGPT to 12 contraception questions, as rated by three gynaecologists using a 3-point Likert scale. The percentage of exact agreement measures interrater reliability. Readability, assessed using FRES and FKGL, ranged from fairly difficult to difficult. Google Gemini scored highest in accuracy, DeepSeek in completeness, and both ChatGPT and DeepSeek in relevance. Across the 36 AI-generated responses evaluated, 47.2% (17/36) contained at least one error or clinically relevant omission. Errors and ambiguities across all models highlight the need for expert supervision.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Evaluating the Performance of Different AI Tools in Generating Contraception Recommendations

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Sik Chun Cheong,
  • Beng Kwang Ng,
  • Pei Shan Lim,
  • Punithanathan A/L Rajoo,
  • Poh Kuan Wong

摘要

Contraception counselling is often limited by time and lack of personalisation, prompting women to seek advice from AI models. This study evaluated the accuracy, completeness, and relevance of responses generated by Google Gemini, DeepSeek, and ChatGPT to 12 contraception questions, as rated by three gynaecologists using a 3-point Likert scale. The percentage of exact agreement measures interrater reliability. Readability, assessed using FRES and FKGL, ranged from fairly difficult to difficult. Google Gemini scored highest in accuracy, DeepSeek in completeness, and both ChatGPT and DeepSeek in relevance. Across the 36 AI-generated responses evaluated, 47.2% (17/36) contained at least one error or clinically relevant omission. Errors and ambiguities across all models highlight the need for expert supervision.