Legal Rationale Generation aims to produce coherent and logically rigorous explanations for judicial decisions based on case facts. Recently, Large Language Models (LLMs) have been explored to improve the performance of generation. While LLMs show promise, existing methods struggle to achieve satisfactory results. For example, most approaches rely on fine-tuning LLMs, making them difficult to implement in resource-constrained scenarios. Besides, directly prompting LLMs to generate rationales often results in a lack of legal grounding due to insufficient legal expertise. To this end, in this paper, we propose a Precedent Retrieval-based In-Context Learning (PRICL) for legal rationale generation framework. This framework retrieves precedents that are prior cases with similar factual descriptions, and then uses them as demonstrations to guide LLMs in generating legal rationales. However, PRICL may still face the shortcut learning problem, where the model relies on spurious correlations rather than deeply understanding the logical relationships between case facts and legal rationales in demonstrations. To mitigate this, we further introduce Meta-PRICL, a method that leverages LLMs to autonomously generate and refine meta-reasoning paths for rationale generation. Extensive experimental results on a real-world dataset clearly validate the effectiveness of our proposed method. Code is available at https://github.com/yuelinan/Codes-of-PRICL .