On Semantic Association Capabilities of GPT LLMs in a Game of Word Associations
摘要
Large Language Models (LLMs) excel at language pattern recognition and prediction of words. The current largest models, such as OpenAI’s Generative Pretrained Transformer (GPT) models, have attained unprecedented levels of success in their ability to perform a variety of tasks, reaching the level of humans in some areas. This paper focuses on an LLM’s core strength—its linguistic predictive capabilities—and how effectively it can be leveraged to find semantic associations between different words. A game of word associations is played with GPT models, in which a set of four words is provided to the player, who has to correctly guess the fifth word that is semantically related to all four words in possibly different contexts. The four provided words are not necessarily semantically related to each other. The testing is done in Serbian, with data obtained from the “associations” game segments of the Serbian national TV quiz “Slagalica” being used to measure the effectiveness of GPT models—notably the currently most powerful GPT-4o model—as players of this game. In a round of the game, ten attempts are given to the model to guess the correct word. It is found that GPT models can discover the correct answer in the majority of the rounds played, with a notable portion of answers being guessed correctly on the first try by the GPT-4 family of models.