Traditional knowledge distillation (KD) transfers knowledge from a pre-trained cumbersome teacher model to a compact student model. In order to reduce the cost of pre-training, online KD removes the teacher and employs several lightweight students to learn from each other. However, existing online KD methods over-estimate the positive facilitation. We argue that over-imitation can lead to homogenization among students, while diversity is far from being explored. In this work, we propose a novel Collaboration and DIversification (CoDI) framework. Concretely, we introduce a class-wise ensemble with dynamic attention in collaboration learning. We emphasize the student identity and introduce non-target diversity to mitigate homogenization. Furthermore, to eliminate conflicts between collaboration and diversification, we bring out a truncated optimization. Experimental evaluations demonstrate the superiority of the proposed CoDI over state-of-the-art methods.