GeoER: A Challenging Benchmark for Geometric Element Recognition
摘要
Recent advancements in Large Multimodal Models (LMMs) have enabled them to tackle complex visual-mathematical reasoning tasks. However, their ability to recognize geometric elements remains underexplored. To address this gap, we introduce GeoER, a novel benchmark designed for evaluating LMMs on Geometric Element Recognition. GeoER consists of 1,080 diverse geometric diagrams sourced from primary and secondary school exams, competitions, and textbooks, ranging from simple geometric shapes to complex combinations. Each diagram is paired with four questions, resulting in 4,320 visual-question-answer pairs. Unlike existing benchmarks focusing on higher-level cognition, GeoER emphasizes recognizing geometric elements through a “simple but interesting” counting task. Evaluation of 12 prominent LMMs, including GPT-4o and Claude 3.5 Sonnet, reveals that these models still struggle with even seemingly simple tasks. Notably, the top-performing model achieved an overall accuracy of only 53.0%, far below human-level performance. These findings will inspire the development of expert-level multimodal foundational models.