Large language models (LLMs) have shown impressive performance in reasoning benchmarks with the emergence of Chain-of-Thought (CoT), particularly in multi-choice question (MCQ). However, current works equally resolve questions regardless of the problem-solving difficulty, leading to an excessive focus on simple items while insufficient attention on intricate ones. To address this challenge, we propose a simple yet effective strategy, Divide and Conquer Reasoning (DCR), to enhance the LLMs’ capability for solving MCQs, as inspired by human beings using heuristics to first categorize tasks and then handle them separately. In particular, we first divide questions into two subsets based on confidence score \(\mathcal{C}\mathcal{S}\) , which is estimated by statistical frequency of generated answers. Subsequently, we propose Filter Choices based Reasoning (FCR) to conquer subset with low \(\mathcal{C}\mathcal{S}\) . Our experiments on three tasks across nine datasets demonstrate that DCR achieves comparable accuracy to SOTA with only 54% overhead and further improves accuracy by 1.56% using 85% of the resources. The code is at https://github.com/AiMijie/DCR .