Business Activity Classification Extraction from Commercial Footage: A Multimodal LLM Approach Based on CNAE (Comparable to NACE and NAICS)
摘要
We employ Brazil’s Economic Activity Classification (CNAE), analogous to European NACE and US NAICS, to develop an experiment for the automatic identification of business activity classification from commercial footage using Multimodal Language Models (LLMs). Although our experiments focus on the Brazilian CNAE, the methodology is applicable to equivalent international systems. Inspired by the Neptune pipeline, we introduce two approaches for models without direct video input (Neptune Image Pipeline and IU Image Pipeline) and a simplified approach (IU Lens Video Pipeline) for models supporting direct video input. The experiments, conducted in 16 videos (approximately 2 hours), revealed that the IU Lens Image Pipeline, which incorporates a consolidation step, significantly outperforms the Neptune Image Pipeline (e.g., gpt-4.1: 89% vs. 22% accuracy), suggesting potential improvements over the original Neptune pipeline. Furthermore, an ensemble voting strategy improved the full accuracy (subclass level), increasing the best individual performance from 46% to 53%, although with some trade-off in upper level accuracy. These results highlight the potential of LLMs for automating critical processes in fiscal management and risk analysis in different regulatory frameworks.