This Looks Like that and that and that: Multi-objective Optimization for Diverse Prototype Learning
摘要
Prototypical Part Networks (ProtoPNets) provide interpretable image classification through prototype-based explanations. However, existing methods suffer from prototype collapse, where multiple prototypes converge to similar patterns, limiting both interpretability and performance. We present ProtoPDiv, a novel extension that explicitly optimizes for prototype diversity through four complementary mechanisms: (1) margin-based diversity loss preventing prototype clustering, (2) orthogonality constraints ensuring prototypes capture independent features, (3) attention-guided learning to focus prototypes on semantically rich regions, and (4) entropy regularization promoting uniform prototype utilization. Our three-phase training strategy progressively introduces these mechanisms while maintaining classification performance. Experiments on CUB-200-2011 demonstrate that ProtoPDiv achieves 93.90% accuracy—a 7.46% improvement over ProtoPNet—while utilizing 28.9% more unique training images for prototype extraction. Visual analysis confirms that ProtoPDiv learns semantically diverse prototypes capturing varied poses, scales, and object parts, addressing the fundamental limitation of prototype collapse in existing methods.