To enable edge-only continual learning on a single edge device, it is important to reflect on the remarkable learning capability of human beings during their lifespan. In particular, with more tasks being learnt, the edge device is expected to be able to learn a new task more easily by leveraging the accumulated knowledge from old tasks (forward knowledge transfer), and also further improve the learning performance of old tasks based on the gained knowledge of related new tasks (backward knowledge transfer). Therefore, in this chapter, we will explore CL algorithm designs for more efficient on-device learning by enhancing both forward and backward knowledge transfer across different tasks.