Price Equilibrium Routing: A Lightweight Framework for Expert Selection in Mixture-of-Experts
摘要
Expert collapse, where a few experts dominate token routing while others remain underutilized, represents a fundamental challenge in Mixture-of-Experts (MoE) models for multimodal learning. Current load balancing approaches rely on auxiliary losses that may conflict with primary task objectives and require careful hyperparameter tuning. We propose Price Equilibrium Routing (PER), a market-inspired framework that addresses expert imbalance through dynamic pricing. Each expert maintains a price that adjusts based on its utilization: overused experts become more expensive and less likely to be selected, while underused ones become cheaper and more attractive. This creates self-regulating behavior that balances expert utilization without explicit constraints. PER is applied as a preprocessing step that adjusts utility scores prior to routing, making it compatible with a wide range of MoE architectures. We evaluate PER across clinical prediction tasks (MIMIC-III/IV) and sentiment analysis (CMU-MOSI), demonstrating improved expert utilization balance with coefficient of variation reductions. PER achieves consistent performance gains across diverse tasks, demonstrating its effectiveness in balancing expert utilization while preserving task accuracy. Overall, this work highlights the promise of economic principles for stable and interpretable MoE routing.