Low Earth Orbit (LEO) satellite networks have gained significant attention due to their extensive coverage and low latency. Unlike terrestrial computing systems, satellite networks are extremely complex, and conventional methods find it challenging to adapt to such a dynamic and intricate environment. Meanwhile as the demand for real-time applications grows, current satellite architectures struggle with efficiently managing the increased need for on-orbit processing. This paper addresses the challenge by proposing an innovative multi-layer LEO satellite edge computing architecture, comprising two tiers of LEO satellites with distinct roles and computational capabilities. Reinforcement learning is leveraged to optimize task offloading, with each satellite acting as an agent capable of consulting a higher-tier agent for improved decision-making in complex scenarios. During implementing our algorithm, we faced a challenge: rewards from task offloading actions are delayed due to extended waiting and execution times. To tackle the issue of delayed feedback, we propose a experience replay buffer algorithm to adapt reinforcement learning. Experimental results demonstrate a 52% reduction in average latency and a 5% decrease in energy consumption per satellite, highlighting the efficiency and practicality of the proposed architecture.