Adversarial distillation has been proven effective in enhancing the adversarial robustness of small models deployed on resource-constrained devices. However, the gap in capacity and complexity between the teacher and student models makes it challenging for the student model to simulate the logits distributions of teacher model, which reduces its learning effectiveness. To address the limitation in existing adversarial distillation methods during robustness transfer, we propose a novel method called Structure-Guided Adversarial Distillation (SGAD). Specifically, SGAD employs multiple teacher models, including an adversarial teacher model as the base model and two additional teacher models by swapping the classification heads between the base teacher model and the student model, allowing the student model to receive diverse supervision signals. Furthermore, SGAD enables selective learning from the correct predictions of the teacher model by introducing an indicator function in the KL divergence loss during distillation, effectively alleviating the mismatch problem in logits distribution. Extensive experiments on two benchmarks demonstrate that SGAD effectively improves the robustness accuracy of student model, while maintaining a good balance between clean performance and adversarial robustness.