Scholarlink

This paper explores how effectively large language models (LLMs) can extract cause-effect (C-E) relationships from Hindi agricultural discourse-a low-resource and domain-specific setting. We compile and manually annotate transcribed Hindi speech data from Television Programme like Krishi Darshan videos and then apply four LLMs: ChatGPT, DeepSeek, Claude, and Copilot- using prompt-based approaches to identify C-E pairs. Due to the implicit and nuanced nature of causal expressions in natural language, the task presents significant challenges. To assess model performance, we conduct both quantitative and qualitative evaluations, incorporating metrics such as plausibility, causal correctness, faithfulness, and a customised weighted average. Among the models, ChatGPT demonstrates the most consistent performance in generating contextually valid and semantically coherent C-E pairs, though it still misses a substantial portion of true positives. DeepSeek and Claude yield moderate results, while Copilot often fails to maintain causal and linguistic alignment. Overall, the findings emphasize the potential and current limitations of LLMs in causal reasoning for low-resource, domain-focused language tasks.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Evaluating LLMs for Causal Extraction from Transcribed Hindi Agricultural Data

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Ayush Patel,
  • Gunjan Ansari,
  • Md. Tauseef Qamar,
  • Imtisal Sahil Ansari,
  • Zaheer Kareem Ansari,
  • Chandni Saxena

摘要

This paper explores how effectively large language models (LLMs) can extract cause-effect (C-E) relationships from Hindi agricultural discourse-a low-resource and domain-specific setting. We compile and manually annotate transcribed Hindi speech data from Television Programme like Krishi Darshan videos and then apply four LLMs: ChatGPT, DeepSeek, Claude, and Copilot- using prompt-based approaches to identify C-E pairs. Due to the implicit and nuanced nature of causal expressions in natural language, the task presents significant challenges. To assess model performance, we conduct both quantitative and qualitative evaluations, incorporating metrics such as plausibility, causal correctness, faithfulness, and a customised weighted average. Among the models, ChatGPT demonstrates the most consistent performance in generating contextually valid and semantically coherent C-E pairs, though it still misses a substantial portion of true positives. DeepSeek and Claude yield moderate results, while Copilot often fails to maintain causal and linguistic alignment. Overall, the findings emphasize the potential and current limitations of LLMs in causal reasoning for low-resource, domain-focused language tasks.