Applying Generative Neural Networks to Extract Argument Relations from Scientific Communication Texts
摘要
The study explores methods for extracting argument relations from texts using large generative language models. Experiments were conducted on a Russian-language corpus of texts related to the field of scientific communication. Prompt-engineering methods were applied, with prompts developed using various tactics. The Mistral-7B was employed as the generative model. The task of extracting argumentative links was formulated as a binary classification problem of the existence/non-existence of a link between two statements. In constructing the dataset, the data were balanced. Positive examples included statements that were part of a single argument (premise, conclusion), while negative examples were generated from statements in the same paragraph for each positive example. Two methods of creating instructions were considered: using ChatGPT and an expert approach using the Chain-of-Thoughts tactic. The best solutions were obtained based on instructions composed by an expert and including context for each statement of one paragraph size. Instructions generated by ChatGPT, while producing comparable results, oftentimes returned incorrect responses. An experimental study was also conducted on an approach, in which the argumentation scheme is predicted immediately, allowing for more precise information about the type of relation to be included in the prompt. This task was also formulated as a binary classification problem. The two most frequent schemes in the examined corpora, “Expert Opinion” and “Example”, were explored.