Federated learning (FL) faces significant challenges when applied to Internet of Things (IoT) environments, including node heterogeneity, high communication overhead, and data privacy concerns. To address the above challenges, we first propose a Federated Learning with Knowledge Distillation (FLwKD) architecture that enables collaborative training among heterogeneous IoT nodes. Building on this architecture, we develop a concrete privacy-preserving FLwKD scheme. Our scheme supports node heterogeneity by allowing each IoT node to adopt a model tailored to its resource capacity. Communication overhead is significantly reduced by exchanging soft label predictions instead of full model parameters/model updates. Data privacy is ensured through threshold homomorphic encryption, which protects soft label predictions during aggregation without revealing individual outputs—even in the presence of partially colluding nodes. Extensive experiments demonstrate that our scheme achieves high model accuracy with significantly reduced communication overhead, making it well-suited for IoT deployments.