This chapter introduces a data-efficient hierarchical reinforcement learning (HRL) framework designed for high-dimensional continuous control tasks. The framework involves optimizing both higher-level and lower-level policies using the same set of samples. Our algorithm demonstrates exceptional data efficiency in achieving optimal performance across four MuJoCo tasks and a simulated robotic assembly environment. Furthermore, the effectiveness of the proposed framework is validated through a real-world robotic dual PiH assembly task.