This paper explores relation classification as a step toward extracting structured information from unstructured Portuguese text. We evaluate prompt-based approaches using generative large language models and compare them with fine-tuned BERT and DeBERTa models, which serve as the baselines in this study. Experiments are conducted on two Portuguese-language datasets: RelEx-PT, a custom sentence-level dataset created by aligning Wikidata triples with Wikipedia sentences, and a second consisting of multi-sentence texts annotated with multiple triples. Results show that, while prompt-based methods offer flexibility and solid performance, they are still not sufficient to overtake fine-tuning, as the latter yields significantly better results in identifying relation types, even widening the gap in more complex task settings.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Prompting LLMs for Relation Classification in Portuguese: Is It Worth It?

  • Tomás Pinto,
  • Bruno Ferreira,
  • Catarina Silva,
  • Hugo Gonçalo Oliveira

摘要

This paper explores relation classification as a step toward extracting structured information from unstructured Portuguese text. We evaluate prompt-based approaches using generative large language models and compare them with fine-tuned BERT and DeBERTa models, which serve as the baselines in this study. Experiments are conducted on two Portuguese-language datasets: RelEx-PT, a custom sentence-level dataset created by aligning Wikidata triples with Wikipedia sentences, and a second consisting of multi-sentence texts annotated with multiple triples. Results show that, while prompt-based methods offer flexibility and solid performance, they are still not sufficient to overtake fine-tuning, as the latter yields significantly better results in identifying relation types, even widening the gap in more complex task settings.