With the rapid development of large language models (LLMs), their remarkable reasoning capabilities have shown potential across various domains. However, most existing studies on LLMs in graph learning focus primarily on tasks involving semantic-rich graphs, where the model’s performance depends on semantic information rather than the intrinsic properties of graph structures. To address this limitation, we propose LLM4MGT, a benchmark with multiscale graph tasks to systematically investigate the capabilities of LLMs in understanding pure graph structures. First, we introduce hierarchical categories of graph-related problems and propose a more coherent and structured benchmark to comprehensively evaluate the boundaries of LLMs’ understanding of graph structures. Second, through extensive experiments on existing commercial APIs, we identify significant issues, including structural hallucinations and reasoning errors, when LLMs process graph inputs. These errors propagate and result in incorrect outputs for subsequent tasks. To mitigate these issues, we develop two simple yet effective prompt strategies that improve LLMs’ ability to process and reason over graph structures while retaining context reasoning capabilities. Despite these advances, LLMs still struggle with complex graph structures and tasks, highlighting the need for further research.