This chapter addresses the foundational challenge of constructing student-teacher architectures in KD. Traditional KD frameworks rely on pre-trained teacher models to guide smaller student networks, but face two critical limitations: (1) the two-stage training process incurs excessive computational overhead, particularly prohibitive for modern large-scale models, and (2) performance instability arises from suboptimal teacher selection, where high-accuracy teachers may paradoxically degrade student performance. To solve these issues, we propose self-distillation as an innovative paradigm that eliminates external teacher dependency. This approach introduces shallow classifiers at intermediate layers to form a multi-exit architecture, enabling intra-model knowledge transfer through hierarchical layer-wise supervision. These findings position self-distillation as a unified solution addressing computational efficiency, deployment flexibility, and performance stability in KD frameworks.