NoRMA: A Multi-agent Communication-Centric Dataset for Enhanced Customs Nomenclature Classification
摘要
Large Language Models-based agents can offer some entry-level consulting in customs services either to importers or exporters. Nevertheless, current general-domain benchmark datasets cannot fully capture the complexity of real-world customs awareness for precise decision-making. To address this, Multi-Agents Systems (MAS) have arisen as a very promising approach to decrease the likelihood of LLMs’ hallucination when acting as artificial intelligence assistants. Given that the Harmonized System (HS) code determination is an essential element for an accurate cost estimation, customs declaration and regulatory compliance for any kind of firm, this paper introduces NoRMA (Nomenclature oRganization for Multi-agent Assistance) and NoRMA-base, two novel high-quality customs datasets consisting of 3424 curated entries each, originally obtained from the official Moroccan Customs and Indirect Tax Administration (Administration des Douanes et Impôts Indirects, ADII), focusing on two chapters: organic chemicals and inorganic chemicals. Subsequently adapted to be integrated directly with Multi-Agent Systems (MAS) for evaluating and fine tuning purposes. Initial evaluation with heavyweight LLMs such as GPT-4o, DeepSeek R1 671B, and LLaMA 3 405B shows they achieve only 55% accuracy on the first 100 questions, justifying the need for a domain-specific fine tuning datasets. To the best of our knowledge, few studies analyze the feature-level combination of LLMs and Multi-Agent systems state-of-the-art for customs nomenclature classification, which amplifies interest in our paper and its findings. The datasets are publicly available https://huggingface.co/datasets/hichambht32/NoRMA .