A prominent approach in Knowledge Base Question Answering (KBQA) is semantic parsing, which converts natural language questions into logical forms (LFs). However, existing methods often face challenges in improving the efficiency and quality of generated LFs. In this paper, we propose GOSP, a granularity-optimized SPARQL generation framework. Firstly, GOSP directly generates SPARQL queries to avoid conversion loss between generated LFs and the final executable query. Secondly, GOSP generates SPARQL queries through three progressively optimized granularities: domain-level prediction with a mask vector to constrain domain within candidates, inferential chain-level prediction guided by the predicted domain to align with knowledge base structures, and execution-level SPARQL generation. Finally, GOSP introduces a dual-path fallback mechanism to enhance the execution-level generation, addressing the issue of empty results from SPARQL execution. Experimental results show that GOSP achieves new state-of-the-art performance on both WebQSP and CWQ benchmarks. Additionally, the results highlight the superiority of SPARQL over other LFs and validate the effectiveness of granularity optimization.